Exploiting Database Technology for Expressive and Efficient Complex Event Processing

Recent research [34] shows that, provided a careful design, complex event processing (CEP) on top of a modern DBMS can compete in many cases with specialized DSMS which have been considered superior for a long time [8]. There are mainly two reasons why building complex event processing on top of an existing DBMS is desirable. First event processing profits form sophisticated techniques and algorithms developed during decades of database research. Second new applications are demanding more functionality, particularly the integration of static and stateful data, but also more expressive event queries and the maintenance and later analysis of logs. These new requirements force specialized DSMS to actually re-implement common database functionality. Starting from that point this doctoral project aims at building an expressive and efficient complex event processing system using a DBMS. For this we first define Temporal Stream Algebra TSA which is an generalization of relational algebra to so-called temporal streams (actually relational algebra is an extremal case of TSA). Second we develop EventMill, a system for efficiently evaluating TSA expressions using a database engine. TSA and Event-Mill are designed to meet the requirements from new ambitious applications of complex event recognition like emergency management in large infrastructures. This includes expressive temporal relations, flexible grouping and aggregation, the integration of static and stateful non-event data, the simultaneous use of multiple timestamps and time models, e.g. application and system time, selective logging, subquery sharing, operator reordering, automatic garbage collection, situation dependent query prioritization, bulk-wise, asynchronous and distributed processing, publish-subscribe dissemination and fault resistance. The doctoral project is part of the international research project EMILI on using CEP for novel forms of emergency management in large infrastructures. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DEBS 2011 Yorktown Heights, NY USA Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00.